Video recording apparatus, multiplexing method, picture encoding method, program, and recording medium for variable frame-rate recording

ABSTRACT

A video recording apparatus provides a slow motion effect or a quick motion effect by digitally converting the frame-rate instead of by changing the playback speed of the film, and using few recording media without requiring any special device at playing. A picture encoding part encodes only valid frames by changing a rate-controlling method so that a predetermined playing frame-rate is obtained at a standard data-rate at playing. And a TS multiplexing part multiplexes by converting a frequency of a standard STC clock by only a ratio of valid frame-rate and playing frame-rate, and generating values for PCR, PTS, and DTS.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video recording apparatus such as avideo camera, and more specifically to a video recording apparatussuitable for providing a slow motion effect or a quick motion effect ina movie by digitally, instead of using a film, converting theframe-rate, and a multiplexing method, a picture encoding method, aprogram, and a recording medium used for configuring the video recordingapparatus.

2. Related Art of the Invention

Conventionally, movies have been taken and edited with a film.Conventional film based video recording apparatus provides a slow motioneffect or a quick motion effect in a movie by changing a recordingframe-rate with fast forwarding or slow playing of the film.

The advance of semiconductor technology, computer technology, andhigh-density recording technology has boosted picture quality andperformance of a broadcasting apparatus system. Now, more and moremovies are being produced with a digital cinema system using a VTR and acomputer-based non-linear editing facility. Such new methods include amethod for digitally providing a slow motion effect and a quick motioneffect with a multi-frame-rate capable imaging device that can set aframe-rate at imaging to any value by controlling a CCD (Charge CoupledDevice) driving method in an imaging part.

For example, “TECHNIQUE CALLED ‘VARIABLE FRAME RATE FUNCTION’”Shashin-Kogyou Publishing Inc. by Katsuyuki Taguchi and Minoru Namikawa,“video α” 2003, April, vol. 19/No. 4, serial number 180 describes a VTRthat can digitally provide a slow motion effect and a quick motioneffect with a multi-frame-rate capable imaging device. This device canvary a frame-rate by one frame/second from four frames/second to 60frames/second to progressively drive a CCD. For simplicity, an imagingframe-rate is X frames/second and a picture signal output from a CCD iscalled an Xp picture signal in the description later. Although aframe-rate can vary by one frame/second in the example of “TECHNIQUECALLED ‘VARIABLE FRAME RATE FUNCTION’”, X needs not be an integer andcan be set any number in the range of real numbers.

A CCD output is Xp picture signals. The signals varied to 60frames/second by so-called pull-down process, which repeats each framefor once or more times, is recorded as 60p picture signals in VTR partincluded in a multi-frame-rate capable imaging device. A VTR usingso-called intra-frame compression, in which compression is completed foreach frame, is used for a VTR here so as to retrieve any frame incompressed video data.

That is to say, the multi-frame-rate capable imaging device is providedwith a view-finder including a liquid crystal panel called EVF(Electronic View Finder). The EVF can typically display only 60p picturesignals and cannot display picture signals at the frame-rate as it shot,for example, at 12 frames/second. Xp picture signals have to besubjected to a pull-down process to be 60p picture signals, which can bedisplayed on EVF, so that EVF can display Xp picture signals of a CCDoutput display. 60p picture signals resulted from the pull-down processis recorded on a VTR.

At playing, 60p picture signals resulted from the pull-down process isplayed from the tape. Conventional video recording apparatus provide aslow motion effect and a quick motion effect by retrieving Xp picturesignals, the signals before the pull-down process, from the 60p picturesignals and changing its time axis to make the frame-rate 24frames/second with a special device called “frame-rate converter”, andrecording the signals as 24p picture signals in a VTR for editing.

SUMMARY OF THE INVENTION

However, conventional video recording apparatus consumed quite manyrecording media as they records 60p picture signals resulted from apull-down process in a VTR, which uses intra-frame compression, wherethey only need to record Xp picture signals.

In addition, a special device called “frame-rate converter” is requiredto perform retrieval of Xp picture signals and time axis conversion tomake 24p picture signals after playing from a VTR.

The present invention intends to provide a video recording apparatus, amultiplexing method, a picture encoding method, a program, and arecording medium for consuming a few recording media without requiringany special device at playing in view of the above mentioned problems.

In order to solve the above problems, the 1^(st) aspect of the presentinvention is a video recording apparatus comprising:

-   -   a multiplexing device operable to multiplex time management        information defining times for decoding and compressed picture        data generated by performing compression encoding of input        picture signals and operable to output a system stream; and    -   a recording device operable to record the system stream on a        recording medium;    -   wherein, when a recording frame-rate of the input picture        signals is different from a preset playing frame-rate for        playing picture signals obtained after recording, playing, and        decoding the system stream, said multiplexing device multiplexes        the time management information corresponding to the preset        playing frame-rate and the compressed picture data.

The 2^(nd) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,comprising a picture encoding device operable to perform compressionencoding of the input picture signals and output the compressed picturedata.

The 3^(rd) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein the compressed picture data is generated by performinginterframe compression on some frames of the input picture signals.

The 4^(th) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein the compressed picture data is generated from the input picturesignals obtained from an imaging device that can vary a frame-rateduring imaging.

The 5^(th) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein said multiplexing device is operable to perform multiplexingcomplying with an MPEG standard, and wherein the time managementinformation includes PCR or SCR used as a timing reference for decodingand outputting (displaying), wherein PTS indicates a timing foroutputting each decoded frame, and DTS indicates a timing for decoding.

The 6^(th) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein said multiplexing device has a clock operable to generate thetime management information, said clock having a frequency which can bevaried from a reference value used when the recording frame-rate and theplaying frame-rate are the same; and

-   -   wherein when the recording frame-rate is different from the        playing frame-rate, said multiplexing device changes a frequency        of said clock from the reference value and uses the changed        frequency to generate the time management information.

The 7^(th) aspect of the present invention is the video recordingapparatus according to the 6^(th) aspect of the present invention,wherein, when the recording frame-rate is X frames/second (X is a realnumber) and the playing frame-rate is Y frames/second, said multiplexingdevice changes a frequency of said clock to X/Y times the referencevalue when X and Y are the same and uses the changed frequency togenerate the time management information.

The 8^(th) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein, when the recording frame-rate is X frames/second (X is an realnumber) and the playing frame-rate is Y frames/second, said multiplexingdevice calculates the time management information so that a differenceof values between pieces of the time management information is X/Y timesa difference of values between corresponding pieces of the referencetime management information, wherein time management informationgenerated when X and Y are the same is the reference time managementinformation.

The 9^(th) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein, when the recording frame-rate is different from the playingframe-rate, said multiplexing device generates the timing referenceinformation with an interval for multiplexing different from a referenceinterval for multiplexing, wherein the reference interval formultiplexing is an interval for multiplexing in case the recordingframe-rate and the playing frame-rate are the same, for timing referenceinformation (PCR) among the time management information.

The 10^(th) aspect of the present invention is the video recordingapparatus according to the 9^(th) aspect of the present invention,wherein, when the recording frame-rate is X frames/second (X is an realnumber) and the playing frame-rate is Y frames/second, said multiplexingdevice generates timing reference information so that the timingreference information (PCR) among the time management information is X/Ytimes a reference interval for multiplexing, wherein the referenceinterval for multiplexing is an interval for multiplexing in case X andY are the same.

The 11^(th) aspect of the present invention is a video recordingapparatus comprising:

-   -   a picture encoding device operable to perform compression        encoding of input picture signals and output the compressed        picture data;    -   a multiplexing device operable to multiplex other information        and the compressed picture data and output a system stream; and    -   a recording device operable to record the system stream on a        recording medium;    -   wherein said picture encoding device is operable to change a        recording data-rate of the compressed picture data from a        reference data-rate and perform compression encoding of the        input picture signals when a recording frame-rate is different        from a playing frame-rate, wherein the reference data-rate is a        playing data-rate of the compressed image data when a recording        frame-rate of the input picture signals and a playing frame-rate        preset in playing picture signals obtained after recording,        playing, and decoding the system stream are the same.

The 12^(th) aspect of the present invention is the video recordingapparatus according to the 11^(th) aspect of the present invention,wherein, when the recording frame-rate is different from the playingframe-rate, said picture encoding device performs compression encodingof the input picture signals so that data-rate during playing of thecompressed picture data approximately matches the reference data-rate.

The 13^(th) aspect of the present invention is the video recordingapparatus according to the 12^(th) aspect of the present invention,wherein, when the recording frame-rate is X frames/second (X is a realnumber), the playing frame-rate is Y frames/second, and the referencedata-rate is R bits/second (R is a real number), said picture encodingdevice performs rate-controlling so that the input picture signals aresubjected to compression encoding at a data-rate of R×(X/Y) bits/second.

The 14^(th) aspect of the present invention is the video recordingapparatus according to the 13^(th) aspect of the present invention,wherein the system stream is a stream complying with an MPEG2 standard;and

-   -   wherein said picture encoding device is operable to set a        picture_rate indicating an input picture frame-rate for        rate-controlling to X and a bit_rate indicating a target rate of        compressed picture data to R×(X/Y), respectively.

The 15^(th) aspect of the present invention is the video recordingapparatus according to the 12^(th) aspect of the present invention,wherein, when the recording frame-rate is X frames/second (X is a realnumber), the playing frame-rate is Y frames/second, and the referencedata-rate is R bits/second (R is a real number), said picture encodingdevice performs rate-controlling so as to perform compression encodingof picture signals whose frame-rate is Y at a data-rate of R bits/secondregardless of a frame-rate of the input picture signals being X.

The 16^(th) aspect of the present invention is the video recordingapparatus according to the 15^(th) aspect of the present invention,wherein the system stream is a stream complying with an MPEG2 standard;and

-   -   wherein said picture encoding device is operable to set a        picture_rate indicating an input picture frame-rate for        rate-controlling to Y and a bit_rate indicating a target rate of        compressed picture data to R, respectively.

The 17^(th) aspect of the present invention is the video recordingapparatus according to the 11^(th) aspect of the present invention,wherein, when the recording frame-rate is different from the playingframe-rate, said picture encoding device changes a data-rate of thecompressed picture data during playing, which corresponds to the playingframe-rate, based on the recording frame-rate.

The 18^(th) aspect of the present invention is a video recordingapparatus comprising:

-   -   a multiplexing device operable to multiplex other information        and compressed picture data generated by performing compression        encoding on picture signals and output a system stream; and    -   a recording device operable to record the system stream on a        recording medium;    -   wherein said multiplexing device is operable to change a        recording system data-rate of the system stream from a reference        data-rate and perform multiplexing when a recording frame-rate        is different from a playing frame-rate, wherein the reference        system data-rate is a playing data-rate of the system stream        when a recording frame-rate of the input picture data and a        preset playing frame-rate of playing picture signals obtained        after recording, playing, and decoding of the system stream are        the same.

The 19^(th) aspect of the present invention is the video recordingapparatus according to the 18^(th) aspect of the present invention,wherein, when the recording frame-rate is different from the playingframe-rate, said multiplexing device performs multiplexing of thecompressed encoding picture signals so that the data-rate of the systemstream during playing, which corresponds to the playing frame-rate,approximately matches the reference system data-rate.

The 20^(th) aspect of the present invention is the video recordingapparatus according to the 19^(th) aspect of the present invention,wherein said multiplexing device is operable to perform the multiplexingso that an output data-rate of the system stream is at Rts×(X/Y)bits/second as said multiplexing device outputs the system stream tosaid recording device when the compressed picture data has beensubjected to compression encoding so that the compressed picture datacan be decoded at R bits/second even if X is different from Y and adata-rate of the system stream is Rts bits/second (Rts is a real numberof Rts>R), wherein the recording frame-rate is X frames/second (X is areal number), the playing frame-rate is Y frames/second, and a playingdata-rate of the compressed picture data when X and Y are the same isreference data-rate R bits/second (R is a real number).

The 21^(st) aspect of the present invention is the video recordingapparatus according to the 18^(th) aspect of the present invention,wherein, when the recording frame-rate is different from the playingframe-rate, said multiplexing device changes a data-rate of the systemstream during playing based on a frame-rate of the input picturesignals.

The 22^(nd) aspect of the present invention is the video recordingapparatus according to the 2^(nd) aspect of the present invention,comprising a valid frame detection device operable to retrieve a validframe flag from the input picture signals by using an input valid frameflag and output the retrieved valid frame to said picture encodingdevice when the input picture signals are picture signals generated byrepeating each frame one or more times and when a valid frame flagindicating a valid frame, which is an original frame for generating saidinput picture signals, is input.

The 23^(rd) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein said multiplexing device is operable to input sound data inaddition to the compressed picture data, and (1) when the recordingframe-rate and the playing frame-rate are the same, said multiplexingdevice multiplexes the input sound data and the system stream, and (2)when the recording frame-rate is different from the playing frame-rate,said multiplexing device does not multiplex the input sound data and thesystem stream.

The 24^(th) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,comprising a sound conversion device, wherein, when sound data is alsoinput in addition to the compressed picture data, and (1) when therecording frame-rate and the playing frame-rate are the same, said soundconversion device outputs the sound data as recorded sound data withoutchanging a sampling frequency of the input sound data, and (2) when therecording frame-rate is different from the playing frame-rate, saidsound conversion device converts the sampling frequency of the inputsound data to a different frequency and outputs the sound data asrecorded sound data; and

-   -   wherein said multiplexing device is operable to multiplex the        output recorded sound data in the system stream.

The 25^(th) aspect of the present invention is the video recordingapparatus according to the 24^(th) aspect of the present invention,wherein said sound conversion device is operable to convert a samplingfrequency of the input sound data to X/Y times the frequency when X andY are different, wherein the recording frame-rate is X frames/second (Xis a real number) and the playing frame-rate is Y frames/second (Y is areal number).

The 26^(th) aspect of the present invention is the video recordingapparatus according to the 1^(st) aspect of the present invention,wherein the system stream comprises a plurality of packets; and

-   -   wherein said recording device is operable to multiplex packet        playing time management information defining times for playing        in the packets included in the system stream, and record it on        the recording medium;    -   wherein said recording device is operable to change packet        playing time information from reference packet playing time        information and multiplex the information when the recording        frame-rate is different from the playing frame-rate;    -   wherein the reference packet playing time information is packet        playing time information when the recording frame-rate and the        playing frame-rate are the same.

The 27^(th) aspect of the present invention is the video recordingapparatus according to the 26^(th) aspect of the present invention,wherein said recording device has a clock operable to generate thepacket playing time information whose frequency can be changed from areference value used when the recording frame-rate and the playingframe-rate are the same; and

-   -   wherein, when the recording frame-rate is different from the        playing frame-rate, said recording device changes a frequency of        said clock from the reference value and uses the changed        frequency to generate the packet playing time management        information.

The 28^(th) aspect of the present invention is the video recordingapparatus according to the 27^(th) aspect of the present invention,wherein, when the recording frame-rate is X frames/second (X is a realnumber) and the playing frame-rate is Y frames/second, said recordingdevice changes a frequency of said clock to X/Y times the referencevalue when X and Y are the same and uses said clock to generate thepacket playing time management information.

The 29^(th) aspect of the present invention is the video recordingapparatus according to the 26^(th) aspect of the present invention,wherein, when the recording frame-rate is X frames/second (X is an realnumber) and the playing frame-rate is Y frames/second, said recordingdevice calculates the packet playing time information so that adifference between values for successive packet playing time informationis approximately X/Y times a difference between values for thecorresponding reference packet playing time information.

The 30^(th) aspect of the present invention is the video recordingapparatus according to the 29^(th) aspect of the present invention,wherein said recording device is operable to determine by selecting adifference of corresponding packet playing time information from aplurality of values and calculate the packet playing time information sothat an average of the selected results is X/Y times a difference of thereference packet playing time information when said recording devicecalculates the difference between values of corresponding packet playingtime information so that the packet playing time information isapproximately X/Y times a difference of successive pieces of thereference packet playing time information.

The 31^(st) aspect of the present invention is a video recordingapparatus comprising:

-   -   a picture encoding device operable to perform compression        encoding of input picture signals obtained from an imaging        device that can vary a frame-rate during imaging and output        compressed picture data; and    -   a recording device operable to record the outputted compressed        picture data on a recording medium;    -   wherein the compressed picture data is recorded on the recording        medium at the frame-rate during imaging; and    -   wherein the compressed picture data recorded on the recording        medium is converted into a previously decided predetermined        frame-rate for playing.

The 32^(nd) aspect of the present invention is a multiplexing methodcomprising:

-   -   multiplexing time management information defining times for        decoding and compressed picture data generated by performing        compression encoding on picture signals and outputting a system        stream; and    -   recording the system stream on a recording medium,    -   wherein, when a recording frame-rate of the input picture        signals is different from a preset playing frame-rate for        playing picture signals obtained after recording, playing, and        decoding the system stream, said multiplexing multiplexes the        time management information corresponding to the preset playing        frame-rate and the compressed picture data.

The 33^(rd) aspect of the present invention is a multiplexing method formultiplexing time management information defining times for decoding andcompressed picture data that has been subjected to compression encodingon X picture signals whose frame-rate is X frames/second (X is a realnumber) so that the X picture signals can be decoded as Y picturesignals whose frame-rate is Y frames/second (Y is a real number) at aplaying bit rate of R bits/second (R is a real number);

-   -   wherein said method comprises multiplexing standard time        management information so that a difference between values of        time management information is X/Y-fold; and    -   wherein when X=Y, a reference time management information is        time management information to be multiplexed on compressed        picture data that has been subjected to compression encoding so        that the X picture signals can be decoded at playing bit rate R        bits/second.

The 34^(th) aspect of the present invention is a multiplexing method formultiplexing time management information defining times for decoding andcompressed picture data that has been subjected to compression encodingso that X picture signals whose frame-rate is X frames/second (X is areal number) can be decoded as Y picture signals whose frame-rate is Yframes/second (Y is a real number);

-   -   wherein said method comprises controlling timing reference        information (PCR) among time management information so that an        interval for multiplying is Y/X-fold.

The 35^(th) aspect of the present invention is a picture encoding methodcomprising:

-   -   performing compression encoding on input picture signals and        outputting the compressed picture data;    -   multiplexing other information and the compressed picture data        and outputting a system stream; and    -   recording the system stream on a recording medium;    -   wherein, said performing compression encoding changes a        recording data-rate of the compressed picture data from a        reference data-rate and performs compression encoding on the        input picture signals when the recording frame-rate is different        from the playing frame-rate, wherein a reference data-rate is a        playing data-rate of the compressed image data when a recording        frame-rate of the picture signals and a preset playing        frame-rate of playing picture signals obtained after recording,        playing, and decoding the system stream are the same.

The 36^(th) aspect of the present invention is a picture encoding methodfor performing compression encoding X picture signals whose frame-rateis X frames/second (X is a real number) so that the X picture signalscan be decoded as Y picture signals whose frame-rate is Y frames/second(Y is a real number) at a playing bit rate Rbits/second (R is a realnumber), said method comprising:

-   -   obtaining a playing frame-rate Y and a playing bit-rate R;    -   inputting a value for imaging a frame-rate X;    -   setting an input picture frame-rate for a rate-controlling        picture_rate to X and a target rate of a compressed picture data        bit_rate to R·(X/Y); and    -   performing rate-controlling of compression encoding by using the        values for the picture_rate and bit_rate.

The 37^(th) aspect of the present invention is a picture encoding methodfor performing compression encoding on X picture signals whoseframe-rate is X frames/second (X is a real number) so that the X picturesignals can be decoded as Y picture signals whose frame-rate is Yframes/second (Y is a real number) at a playing bit rate R bits/second(R is a real number), said method comprising:

-   -   obtaining a playing frame-rate Y and a playing bit rate R from        outside or a memory;    -   inputting a value for imaging a frame-rate X;    -   setting an input picture frame-rate for a rate-controlling        picture_rate to Y and a target rate of a compressed picture data        bit_rate to R; and    -   performing rate-controlling on compression encoding by using        values for the picture_rate and bit_rate.

The 38^(th) aspect of the present invention is a program for use with acomputer, said program comprising:

-   -   an executable code operable to cause the computer to multiplex        time management information defining times for decoding and        compressed picture data generated by performing compression        encoding of picture signals and cause the computer to output a        system stream; and    -   an executable code operable to cause the computer to record the        system stream on a recording medium;    -   wherein, when a recording frame-rate of the input picture        signals is different from a preset playing frame-rate for        playing picture signals obtained after recording, playing, and        decoding the system stream, said executable code operable to        cause the computer to multiplex multiplexes the time management        information corresponding to the preset playing frame-rate and        the compressed picture data.

The 39^(th) aspect of the present invention is a program for use with acomputer, said program comprising:

-   -   an executable code operable to cause the computer to performing        compression encoding of input picture signals and output the        compressed picture data;    -   an executable code operable to cause the computer to multiplex        other information and the compressed picture data and output a        system stream; and    -   an executable code operable to cause the computer to record the        system stream on a recording medium; and    -   wherein said executable code operable to cause the computer to        performing compression encoding is operable to change a        recording data-rate of the compressed picture data from a        reference data-rate and perform compression encoding of the        input picture signals when a recording frame-rate is different        from a playing frame-rate, wherein the reference data-rate is a        playing data-rate of the compressed image data when a recording        frame-rate of the input picture signals and a playing frame-rate        preset in playing picture signals obtained after recording,        playing, and decoding the system stream are the same.

The 40^(th) aspect of the present invention is a program for use with acomputer, said program comprising:

-   -   an executable code operable to cause the computer to multiplex        other information and compressed picture data generated by        performing compression encoding on picture signals and cause the        computer to output a system stream; and    -   an executable code operable to cause the computer to record the        system stream on a recording medium;    -   wherein said executable code operable to cause the computer to        multiplex is operable to change a recording system data-rate of        the system stream from a reference data-rate and perform        multiplexing when a recording frame-rate is different from a        playing frame-rate, wherein the reference system data-rate is a        playing data-rate of the system stream when a recording        frame-rate of the input picture data and a preset playing        frame-rate of playing picture signals obtained after recording,        playing, and decoding of the system stream are the same.

The 41^(st) aspect of the present invention is a computer readablemedium containing a program according to the 38^(th) aspect of thepresent invention.

The 42^(nd) aspect of the present invention is a computer readablemedium containing a program according to the 39^(th) aspect of thepresent invention.

The 43^(rd) aspect of the present invention is a computer readablemedium containing a program according to the 40^(th) aspect of thepresent invention.

The 44^(th) aspect of the present invention is the video recordingapparatus according to the 11^(th) aspect of the present invention,wherein said multiplexing device is operable to input sound data inaddition to the compressed picture data, and (1) when the recordingframe-rate and the playing frame-rate are the same, said multiplexingdevice multiplexes the input sound data and the system stream, and (2)when the recording frame-rate is different from the playing frame-rate,said multiplexing device does not multiplex the input sound data and thesystem stream.

The 45^(th) aspect of the present invention is the video recordingapparatus according to the 18^(th) aspect of the present invention,wherein said multiplexing device is operable to input sound data inaddition to the compressed picture data, and (1) when the recordingframe-rate and the playing frame-rate are the same, said multiplexingdevice multiplexes the input sound data and the system stream, and (2)when the recording frame-rate is different from the playing frame-rate,said multiplexing device does not multiplex the input sound data and thesystem stream.

The 46^(th) aspect of the present invention is the video recordingapparatus according to the 11^(th) aspect of the present invention,comprising a sound conversion device, wherein, when sound data is alsoinput in addition to the compressed picture data, and (1) when therecording frame-rate and the playing frame-rate are the same, said soundconversion device outputs the sound data as recorded sound data withoutchanging a sampling frequency of the input sound data, and (2) when therecording frame-rate is different from the playing frame-rate, saidsound conversion device converts the sampling frequency of the inputsound data to a different frequency and outputs the sound data asrecorded sound data; and

-   -   wherein said multiplexing device is operable to multiplex the        output recorded sound data in the system stream.

The 47^(th) aspect of the present invention is the video recordingapparatus according to the 18^(th) aspect of the present invention,comprising a sound conversion device, wherein, when sound data is alsoinput in addition to the compressed picture data, and (1) when therecording frame-rate and the playing frame-rate are the same, said soundconversion device outputs the sound data as recorded sound data withoutchanging a sampling frequency of the input sound data, and (2) when therecording frame-rate is different from the playing frame-rate, saidsound conversion device converts the sampling frequency of the inputsound data to a different frequency and outputs the sound data asrecorded sound data; and

-   -   wherein said multiplexing device is operable to multiplex the        output recorded sound data in the system stream.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an imaging/recording device witha video recording apparatus of embodiment 1 according to the presentinvention;

FIG. 2 is a diagram of signal waveforms for each part of FIG. 1;

FIG. 3 is a detailed block diagram illustrating video recordingapparatus 2 of the embodiment 1 according to the present invention;

FIG. 4 is a diagram of signal waveforms for respective parts of FIGS. 1and 3;

FIG. 5 is a diagram showing a data capacity of bit buffer 36 and a VBVbuffer capacity allowing for decoding at playing;

FIG. 6 is a chart showing a procedure in rate controlling means 32;

FIG. 7 is a block diagram illustrating a configuration to be added toSTC clock generation means 53;

FIG. 8 is a diagram of signal waveforms for respective parts of FIGS. 1and 3 in the case of slow motion;

FIG. 9 is a diagram showing a data capacity of bit buffer 36 and a VBVbuffer capacity allowing for decoding at playing in the case of slowmotion;

FIG. 10 is a block diagram illustrating video recording apparatus 2 inembodiment 2 according to the present invention;

FIG. 11 is a conceptual diagram of storing packet timing when MPEG2-TSis recorded on recording means 23;

FIG. 12 is an example of a TS packet added with ATS;

FIG. 13 is an example of a circuit for reproducing playing timing for aTS packet added with ATS;

FIG. 14 is a timing chart illustrating operations of respective partswhen imaging frame-rate value X=(8/1.001) fps;

FIG. 15 is a timing chart illustrating operations of respective partswhen imaging frame-rate value X=14.985 fps;

FIG. 16 is a diagram illustrating operations of respective parts whenimaging frame-rate value X=14. 985 fps and a fraction of an intervalbetween TS packets is addressed; and

FIG. 17 is a block diagram illustrating an exemplary configuration ofATS value generation means 90.

PREFERRED EMBODIMENTS OF THE INVENTION

A video recording apparatus, a picture encoding method, a multiplexingmethod, a program, and a recording medium according to the presentinvention can provide a video recording apparatus consuming a fewrecording media by recording only valid frames (X picture signals) withthe above mentioned configuration. They can also provide a videorecording apparatus that does not require any special device at playingas it performs a compression encoding process and a multiplexing processbased on a preset playing frame-rate.

Now, a video recording apparatus, a picture encoding method, and amultiplexing method according to the present invention will be describedwith reference to accompanying drawings.

Embodiment 1

FIG. 1 is a block diagram illustrating an imaging/recording device witha video recording apparatus of embodiment 1 according to the presentinvention.

In FIG. 1; reference numeral 1 denotes an imaging device, 2 denotes avideo recording apparatus, 3 denotes an electronic view finder(hereinafter abbreviated for EVF), 4 denotes a microphone, 5 denotes acontrolling part, 6 denotes 60p picture signals, 7 denotes a valid frameflag for retrieving a valid frame corresponding to Xp picture signalsoutput from CCD 10 from 60p picture signals.

Reference numeral 10 denotes a CCD, which can scan progressively, 11denotes frame memory, 12 denotes a camera processing part for performinga camera process, 13 denotes a CCD driving circuit, and 14 denotes adriving pulse switching circuit for switching to and outputting adriving pulse corresponding to a multi-frame rate.

Reference numeral 20 denotes a valid frame retrieval part for retrievinga valid frame from 60p picture signals and outputting the frame as Xppicture signals, 21 denotes a picture encoding part for performingMPEG-video-compression encoding on Xp picture signals and outputting thecompressed picture data, 22 denotes a TS multiplexing part foroutputting a system stream in the form of MPEG-TS (Transport Stream) bymultiplexing time management information defining times for decoding andother necessary information on input compressed picture data orcompressed sound data, 23 denotes a recording part for recording asystem stream on a recording media, and 24 denotes a sound encodingpart.

Picture encoding part 21 in this embodiment is an example of pictureencoding means of the present invention, TS multiplexing part 22 in thisembodiment is an example of multiplexing device of the presentinvention, and recording part 23 in this embodiment is an example ofrecording device of the present invention.

Next, operations of the embodiment will be described.

Imaging device 1 can vary a frame-rate of an imaging device as aconventional imaging device described in “Description of the RelatedArt” does. Controlling part 5 receives imaging frame-rate value X andplaying frame-rate value Y, which are both set by a user, from a usermanipulation part (not shown), and outputs controlling signalscorresponding to the values to each part. Playing frame-rate value Y is24 frames/second in the description later. In this embodiment, animaging frame-rate for a CCD is considered as X frames/second, andpicture signals output from a CCD is called Xp picture signals asmentioned in “Description of the Related Art”.

Driving pulse switching circuit 14 receives controlling signals fromcontrolling part 5, generates driving pulses needed for imaging Xppicture signals, and outputs the pulses to CCD driving circuit 13. CCDdriving circuit 13 drives CCD 10 by converting the driving pulses fromdriving pulse switching circuit 14 to a predetermined voltage. CCD 10outputs the imaged Xp picture signals to frame memory 11.

FIG. 2 is a diagram of signal waveforms for each part of FIG. 1.Reference numeral 502 a denotes output signals from CCD 10. Althoughoutput signals from CCD 10 appear as actual Xp picture signals in FIG.2, output signals are only required to include picture informationcorresponding to Xp picture signals and a format of synchronizationsignals are not necessarily the same as that of Xp picture signals.

Frame memory 11 outputs 60p picture signals resulted from a pull-downprocess by writing picture information on Xp picture signals from CCD 10and reading out the picture information for the same frame for multipletimes. Camera processing part 12 performs a predetermined process of acamera process on the 60p picture signals as 60p picture signals andoutputs the signals as 60p picture signals indicated by 502 b to EVF 3and valid frame retrieval part 20. EVF 3 can easily display pictureinformation on Xp picture signals by displaying the picture at 60frames/second.

The picture information varies by an Xp picture signal in 60p picturesignals resulted from a pull-down process. As indicated by 502 c, validframe flag 7 for retrieving a valid frame corresponding to Xp picturesignals from 60p picture signals is output to valid frame retrieval part20 at transition between pieces of picture information. Although validframe flag 7 is adapted to be output separately from picture signal 6,it can be multiplexed and output in a vertical blanking period ofpicture signal 6, for example.

Valid frame retrieval part 20 retrieves Xp picture signals from 60ppicture signals by using valid frame flag 7 and outputs the Xp picturesignals to picture encoding part 21. Picture encoding part 21 performsMPEG-video-compression encoding on the input Xp picture signals andoutputs the compressed picture data. The compression encoding isperformed on the Xp picture signal here, while controlling of acompression encoding rate and addition of a header are performed on thebasis of 24 frames/second at playing. If a reference data rate is Rbits/second when pictures are recorded in normal recording mode (i.e.,an imaging frame-rate, and a playing frame-rate are the same (X=24)),picture encoding part 21 performs rate controlling to performcompression encoding Xp picture signals to R·(X/24) bits/second. Theprocess will be detailed later.

On the other hand, sound signals recorded with microphone 4 are inputinto sound encoding part 24, subjected to audio-compression encoding,and the compressed sound data is output. The sound signals can beprocessed in accordance with a frame-rate conversion of picture signals.In other words, where a sampling frequency of sound signals is Fa and areference data-rate after compression encoding is Ra, the samplingfrequency of the sound signals is converted to (X/24) in accordance withthe frame-rate of input picture signals and the sound signal iscompressed by performing a rate-control so that the data-rate of thecompressed sound data will be Ra·(X/24) in performing audio compressionencoding. Instead of this, sound data, which is a sampling frequencyconverted into (X/24) in accordance with a frame-rate of an inputpicture signal, can be multiplexed on input sound signals withoutperforming compression encoding.

That is to say, when the frame-rate of input picture signals is equal tothe frame-rate of a system stream at playing, sound encoding part 24outputs the sound data as recorded sound data without changing thesampling frequency of the input sound data. When the frame-rate of theinput picture signals is different from the frame-rate of the systemstream at playing, sound encoding part 24 may convert the samplingfrequency of the input sound data into another frequency and output thesound data as recorded sound data.

The embodiment is described for the case that compressed sound data isgenerated and multiplexed only when X=24 and compressed sound data isneither generated nor multiplexed otherwise, as a more simplifiedexample of implementation.

In other words, TS multiplexing part 22 inputs sound data as well ascompressed picture data. When the frame-rate of input picture signals isequal to the frame-rate of a system stream at playing, TS multiplexingpart 22 multiplexes the input sound data into the system stream. Whenthe frame-rate of input picture signals is different from the frame-rateof a system stream at playing, TS multiplexing part 22 does notmultiplex the input sound data into the system stream.

TS multiplexing part 22 multiplexes time management information definingtimes for decoding and other necessary information on compressed picturedata input from picture encoding part 21 and compressed sound data inputfrom sound encoding part 24 and outputs a system stream in the form ofMPEG-TS indicated by 502 d.

In 502 d or the like, “I” in “I2”, for example, denotes I picture, i.e.,an encoded picture within a frame (Intra encoded picture), “B” in “B0”,for example, denotes B picture, i.e., a bidirectionally predictiveencoded picture, “P” in “P5”, for example, denotes P picture, i.e.,predictive encoded picture in a forward direction between frames(Predictive encoded picture). Therefore, unlike the conventional videorecording apparatus described in “Description of the Related Art”, thevideo recording apparatus of the embodiment uses compression betweenframes for B pictures or P pictures as well as interframe compression.

A system stream output from TS multiplexing part 22 is recorded inrecording part 22. Time management information includes PCR (ProgramClock Reference) used as a timing reference for decoding and outputting(displaying), PTS (Presentation Time Stamp) indicating a timing tooutput each decoded frame, and DTS (Decoding Time Stamp) indicating atiming for decoding.

When a system stream recorded in recording part 23 of a video recordingapparatus of the embodiment is output to an outside decoding device andthe decoding device decodes the output system stream, STC (System TimeClock) used as an operational reference for the decoding device isgenerated by using PCR. Then, the decoding device is controlled so thatdecoding and displaying is performed at a time specified as PTS/DTS inaccordance with the STC. Therefore, when Xp picture signals aresubjected to compression encoding and multiplexing, values for PCR, PTS,and DTS need to be changed so that the controlling is performed atplaying in the same manner as in the case that a system stream, in which24p picture signals are subjected to compression encoding andmultiplexed, is played. Thus, when a frame-rate of Xp picture signals isdifferent from a preset frame-rate of a system stream at playing to berecorded in recording part 23, TS multiplexing part 22 multiplexes timemanagement information corresponding to a preset frame-rate of a systemstream at playing into compressed picture data. The frame-rate of Xppicture signals of the embodiment is an example of a recordingframe-rate of the present invention and the preset frame-rate of asystem stream at playing of the embodiment is an example of a playingframe-rate of the present invention. This will be detailed later.

At playing, a system stream is played in the same form as 24p picturesignals are played at a reference data-rate as indicated by 502 e.Compressed picture data is also decoded in the same way as for usual 24ppicture signals as indicated by 502 f. Finally, decoded 24p picturesignals are subjected to so-called 2-3 pull-down to a progressive or aninterlace picture at 60 frames/second to comply with interface generallyused between the video recording apparatus of the embodiment and apicture monitor and then output. Therefore, when compression encoding isperformed, picture encoding part 21 has to set “repeat_first_field,top_field_first” in correspondence with this 2-3 pull-down.

FIG. 3 is a block diagram illustrating video recording apparatus 2 ofthe embodiment 1 according to the present invention in more detail. Thesame numerals are given to the same components as those in FIG. 1 andthe detailed descriptions thereof are omitted. Reference numerals 30 and31 denote frame memory for storing frames corresponding to a B pictureof MPEG2 for changing the order of processing the input frame, 32denotes rate controlling means of controlling to make the data rate ofthe compressed picture data approach a target value, 33 denotessubtractor, 34 denotes DCT means, 35 denotes quantization means, 36denotes a bit buffer for temporally storing compressed picture data inorder to output the compressed picture data at a constant rate, 37denotes counter-quantization means, 38 denotes IDCT means, 39 denotesadder, and 40 and 41 denote frame memory for storing reference pictures.Picture encoding part 21 including them performsMPEG2-video-compression-encoding. As MPEG2-Video is a well knownstandard to those skilled in the art, detailed description of respectiveterms based on the standard, such as DCT or quantization, and operationsof associated blocks will be omitted. As a single picture to be encodedis generally called “a picture” in MPEG2 standard, “a picture” and “aframe” are used interchangeably in the following description. Thus, theterms “a frame” and “a picture” mean the same thing here.

Reference numerals 50 and 55 denote PES-making means of generating a PESpacket corresponding to a picture and a sound respectively, 51 and 56denote TS-making means of generating a TS packet by dividing an inputPES packet and multiplexing necessary information, 52 denotesmultiplexing means of multiplexing a TS packet and another necessarypacket for a picture and a sound respectively, and 53 denotes STC clockgeneration means of generating an STC clock used as a reference for atime axis to create time management information. STC clock generationmeans 53 is an example of a clock according to the present invention.Reference numeral 54 denotes an STC controlling circuit for generatingtime management information from an STC clock, while managing timing formultiplexing. TS multiplexing part 22 including them performs MPEG2-TSmultiplexing. As MPEG2-TS is a well known standard to those skilled inthe art, detailed description of respective terms based on thisstandard, such as PES (Packetized Elementary Stream) or TS (TransportStream), and operations of associated blocks will be omitted.

FIG. 4 is a diagram of signal waveforms for respective parts of FIGS. 1and 3. 504 d is a chart illustrating a content of frame memory 30, 504 eis a chart illustrating a content of frame memory 31, 504 f is a chartillustrating a content of frame memory 40, and 504 g is a chartillustrating a content of frame memory 41. In order to limit the framememory for storing frames corresponding to B pictures to two of 30 and31, each frame of Xp picture signals has to be encoded at the same speedas that for 60p picture signals as shown in 504 h.

Now, rate controlling in compression encoding of a picture according tothe present invention will be detailed. As mentioned above, pictureencoding part 21 has to perform rate controlling so that Xp picturesignals are subjected to compression encoding to R·(X/24) bits/second inthe present invention.

FIG. 5 is a diagram showing a data capacity of bit buffer 36 in apicture encoding part and a VBV buffer capacity allowing for decoding atplaying. Rate controlling means 32 has to control a data rate ofcompressed picture data so that a VBV buffer at playing generatesneither overflow nor underflow as shown in 505 e. 505 c is a chartillustrating how each frame is recorded with time during recording. 505d is a chart illustrating how each frame is played with time at playing.When 505 c and 505 d are compared, the time axis changes based on aratio between an imaging frame-rate X and playing frame-rate 24, thoughthe content of data is exactly the same. Therefore, the data-rate in 505c is what the data-rate in 505 d multiplied by X/24.

As a result, a rate at which a data capacity of a bit buffer decreases(be output) in 505 b is what a standard rate of compressed picture dataR multiplied by X/24. On the other hand, a distance between pictures isan inverse number of imaging fame-rate X. Thus, amounts of data to begenerated by performing compression encoding on frames respectively,i.e., the lengths of respective vertical lines (vertical lines 100 and200, 101 and 201) assuming the scale of vertical axes of 505 b and 505 eare the same are equal. That is to say, a target amount of data for eachcompressed frame is the same as when 24p picture signals are compressed(although input picture signals are Xp picture signals) at ratecontrolling means 32.

FIG. 6 is a chart showing a procedure in rate controlling means 32. Atstep S1, playing frame-rate Y (fixed to 24 in this embodiment) andplaying bit-rate R (a standard rate for compressed picture data) areset. Values for Y and R can be determined from inputs with switches oron a GUI (Graphical User Interface) screen, for example, or can be readout from non-volatile memory that has already stored preset values atimplementing the video recording apparatus. Then at step S2, imagingframe-rate value X is set from inputs with switches or on a GUI screen.

At step S3, parameters to be used for controlling actual rates arecalculated. One of them is a picture_rate, which is a frame-rate forpictures input into picture encoding part 21. Imaging frame-rate value Xcan be used for the picture_rate. The other is a bit_rate, which is atarget rate of compressed picture data. The bit_rate can be obtained bymultiplying a playing bit rate R by a ratio between imaging frame-rateand playing frame-rate (X/Y).

Finally at step S4, rate controlling is performed on compressionencoding by using values for Picture_Rate and bit_rate obtained at stepS3. Any method can be used for rate controlling. For example, arate-controlling method in MPEG2 TM5 (Test Model 5), which is publiclyknown and used in standardizing MPEG2, is available. The case where acontrolling method for MPEG2 TM5 is used will be described below.

At rate controlling means 32, a target amount of data to be allocated tothe picture next to a certain GOP (Group of Picture) is set according toan expression shown in Formula 1, for example. In Formula 1, Ti is atarget amount of data to be allocated to I picture next. Tp is a targetamount of data to be allocated to P picture nest. Tb is a target amountof data to be allocated to B picture next. Kp and Kb are constantnumbers depending on processing of quantization means 35. Rgop is aremaining amount of bits allocated to a currently processed GOP. Np andNb are remaining numbers of P pictures and B pictures included in acurrently processed GOP. Xi, Xp and Xb are indicators indicating howpreviously encoded I pictures, P pictures, and B pictures arecomplicated, respectively. As details of the parameters are disclosed inMPEG TM 5, description of them will be omitted. $\begin{matrix}{{{Ti} = {\max\{ {\frac{Rgop}{( {1 + \frac{{Np} \cdot {Xp}}{{Xi} \cdot {Xp}} + \frac{{Nb} \cdot {Xb}}{{Xi} \cdot {Kb}}} )},\frac{bit\_ rate}{8 \cdot {picture\_ rate}}} \}}}{{Tp} = {\max\{ {\frac{Rgop}{( {{Np} + \frac{{Nb} \cdot {Kp} \cdot {Xb}}{{Kb} \cdot {Xp}}} )},\frac{bit\_ rate}{8 \cdot {picture\_ rate}}} \}}}{{Tb} = {\max\{ {\frac{Rgop}{( {{Nb} + \frac{{Np} \cdot {Kb} \cdot {Xb}}{{Kp} \cdot {Xb}}} )},\frac{bit\_ rate}{8 \cdot {picture\_ rate}}} \}}}} & \lbrack {{Formula}\quad 1} \rbrack\end{matrix}$

Formula 2 is provided as an initial value of Rgop at the top of thesequence for encoding in Formula 1. The initial value is updated eachtime a picture is encoded thereafter. N is the number of picturesincluded in a first GOP in a sequence. $\begin{matrix}{{Rgop\_ ini} = \frac{{bit\_ rate} \cdot N}{picture\_ rate}} & \lbrack {{Formula}\quad 2} \rbrack\end{matrix}$

Therefore, Ti, Tp, and Tb obtained by Formula 1 change according to abit_rate, which is a target rate for compressed picture data, and apicture_rate, which is a frame rate. When rate controlling means 32 isimplemented in accordance with MPEG2 TM5 (using Formula 1 and Formula2), buffer controlling shown in 505 b can be easily provided by makingbit_rate R·(X/24) and picture_rate X.

When the frame-rate of input picture signals is X frames/second, apreset frame-rate at playing of a system stream is Y frames/second, andthe reference data-rate at playing of a compressed picture data includedin a system stream is R bits/second, picture encoding part 21 thusperforms rate-controlling so that input picture signals are subjected tocompression encoding at a data-rate of R×(X/Y) bits/second.

It is indicated that a target amount of data for each compressed frameis the same as when 24p picture signals are compressed (although inputpicture signals are Xp picture signals) in FIG. 5. Accordingly, aprocess at step S3 can be changed to another process of settingpicture_rate=Y and bit_rate=R. In the above mentioned rate controllingfor MPEG2 TM5, every expression associated with rate controlling changesdepending on a proportion of bit_rate/picture_rate and both of them areused, the same result can be obtained when bit_rate is R andpicture_rate is Y, as shown in Formula 3. $\begin{matrix}\begin{matrix}{\frac{bit\_ rate}{picture\_ rate} = \frac{R \cdot ( {X/Y} )}{X}} \\{= \frac{R}{Y}}\end{matrix} & \lbrack {{Formula}\quad 3} \rbrack\end{matrix}$

That is to say, when the frame-rate of input picture signals is Xframes/second, the preset frame-rate at playing of a system stream is Yframes/second, and the reference data-rate at playing of a compressedpicture data included in a system stream is R bits/second, pictureencoding part 21 can obtain the same result as mentioned above eventhough it performs rate controlling so as to perform compressionencoding on picture signals whose frame-rate is Y at a data-rate of Rbits/second, regardless of the frame-rate of input picture signals beingX.

In other words, when the frame-rate of input picture signals isdifferent from the frame-rate at playing, picture encoding part 21performs compression encoding on the input picture signals so that thesignals virtually matches with a reference data-rate at playing ofcompressed picture data, corresponding to the frame-rate at playing.Thus, picture encoding part 21 can control data-rate of compressedpicture data so that a VBV buffer generates neither overflow norunderflow at playing.

Next, generation of time management information of the embodimentaccording to the present invention will be detailed. As mentioned above,this embodiment requires changing an interval for multiplexing PCR, PTS,and DTS, while changing values for PCR, PTS, and DTS so that the samecontrolling can be performed as in the case that a system stream, inwhich 24p picture signals have been subjected to compression encodingand multiplexing at playing, is played.

When MPEG2-TS encoded, multiplexed, and recorded as shown in 505 c isplayed and decoded as shown in 505 d, time intervals between PCRs, PTSs,and DTSs which are multiplexed at 505 c are 24/X times their timeintervals in 505 d. Thus, when the frame-rate of input picture signalsis X frames/second and the preset frame-rate of a system stream atplaying is Y frames/second, the time interval for multiplexing PCR, PTS,and DTS is Y/X times a reference time interval when it is recorded at Xframes/second, where a time interval for the same value of X and Y is areference time interval. On the other hand, these values are generatedby using a count at the timing required for counting STC clock, which isused as a reference for a time axis at TS multiplexing part 22, and formultiplexing. Therefore, when the interval for multiplexing is24/X-fold, the interval between values is also 24/X-fold, which leads tosome problems.

Therefore, in the present invention, values for PCR, PTS, and DTS areadjusted. One of the ways is available by generating PCR, PTS, and DTS(an interval between whose values is 24/X-fold) leaving an STC clock ata usual frequency, then multiplying these values by X/24 with acalculator or software, and multiplexing the result. That is to say,when the frame-rate of input picture signals is X frames/second and thepreset frame-rate of a system stream at playing is Y frames/second, TSmultiplexing part 22 calculates time management information andmultiplexes so that a difference of values between pieces of timemanagement information is X/Y times a difference of values betweenpieces of reference time management information, where time managementinformation generated as X and Y are the same is reference timemanagement information. STC controlling circuit 54 for performing thisprocess is easily available by adding a calculator or a CPU to outputstages of PCR, PTS, and DTS.

As an interval for multiplexing PCR is defined as 100 ms at the maximumin the MPEG2 standard, an interval for multiplexing needs to becontrolled to comply with this restriction at playing bit_rate R. Thisis available by controlling STC control circuit 54 so that an intervalfor multiplexing PCR is a time uniformly multiplied by 24/X in an actualtime. That is to say, when the frame-rate of input picture signals is Xframes/second and the preset frame-rate of a system stream at playing isY frames/second, TS multiplexing part 22 adds PCR so that an intervalfor multiplexing PCR is Y/X times a reference interval for multiplexing,where an interval for multiplexing as X and Y are the same is areference interval for multiplexing.

Another way to comply with the above mentioned restriction is availableby multiplying a STC clock used for generating values for PCR, PTS, andDTS by X/24. In order to generate this STC clock, a circuit needs to beadded to STC clock generation means 53.

FIG. 7 is a block diagram illustrating a configuration to be added toSTC clock generation means 53. Reference numeral 60 denotes controllingmeans of controlling frequency dividing ratios P, Q, and R for frequencydividers 61, 65, and 66, 61 denotes a frequency divider for dividing afrequency of a standard STC clock to be input by 1/Q, 62 denotes a phasedifference detector, 63 denotes a loop filter, 64 denotes a VCO, and 65denotes a frequency divider for dividing a frequency of an output fromVCO 64 by 1/P. Controlling means 60, frequency divider 61, phasedifference detector 62, loop filter 63, VCO 64, and frequency divider 65comprise PLL. Reference numeral 66 denotes a frequency divider fordividing a frequency of an output from VCO 64 by 1/R.

With a configuration shown in FIG. 7, a frequency of STC clock (27 MHz)can be changed to {27 MHz·(P/Q)}/R. Therefore, values for PCR, PTS, andDTS needed for multiplexing MPEG2-TS of 505 c can be generated bymultiplying a frequency of STC clock by X/24 with this configuration.

That is to say, when the frame-rate of input picture signals is Xframes/second and the preset frame-rate of a system stream at playing isY frames/second, TS multiplexing part 22 changes a frequency of STCclock to X/Y times a reference value as X and Y are the same (27 MHz)and generates respective values for PCR, PTS, and DTS with the changedSTC clock. Thus, when the frame-rate of input picture signals isdifferent from the preset frame-rate of a system stream at playing, TSmultiplexing part 22 may change a frequency of STC clock from areference value (27 MHz) and generate respective values for PCR, PTS,and DTS with the changed STC clock.

Although FIGS. 4 and 5 illustrate the case of triple quick motion atplaying where imaging frame-rate X=8, slow motion can also be providedat playing by increasing imaging frame-rate X higher than 24frames/second of playing frame-rate as shown in FIGS. 8 and 9. In thiscase, it is apparent by comparing FIGS. 4 and 8, and FIGS. 5 and 9, slowmotion can be provided at playing by applying the above mentionedconfiguration and operations, as an imaging frame-rate X and playingframe-rate 24 are only different in size.

The embodiment 1 has thus provided a video recording apparatus, whichconsumes a few recording medium by only recording valid frames (Xppicture signals). The embodiment 1 can save recording medium as it canachieve high compression ratio by using MPEG2, which is a technology ofencoding between frames, instead of encoding within a frame for picturecompression encoding. The embodiment 1 can also provide a videorecording apparatus that requires no special device, as the apparatusperforms a compression encoding process and a multiplexing process basedon a preset playing frame-rate (24 frame/second) and signals can beplayed at the same frame-rate (24 frames/second) and at the samedata-rate R as they were recorded in a usual manner.

Needless to say, although the embodiment 1 is described by assuming thatimaged Xp picture signals are pull-downed to 60p picture signals andinput into a video recording apparatus and that a playing frame-rate is24 frames/second, picture signals pull-downed and converted into any Nframes/second can be input into a video recording apparatus and aplaying frame-rate can be processed at M frames/second.

Embodiment 2

FIG. 10 is a block diagram illustrating video recording apparatus 2 inembodiment 2 according to the present invention. The same numerals aregiven to the same components as those in FIGS. 1 and 3, and the detaileddescriptions thereof are omitted. Reference numeral 90 denotes ATS valuegeneration means, 91 denotes ATS multiplexing means, and 92 denotes arecording media controlling part.

If an actual available transfer rate is high when MPEG2-TS istransferred from TS multiplexing means 22 to recording means 23, a TSpacket will be transferred intermittently. In order to record MPEG2-TSefficiently on a recording medium, unnecessary information such as Nullpacket in an actual transport stream may be removed. In this case,timing for a TS packet to be input and output to and from recordingmeans 23 needs to be somehow recorded on a recording medium. Bru-raydisk or the like uses Arrival Time Stamp (ATS) for this purpose.

Effects of ATS on general recording and playing will be described below.FIG. 11 illustrates a concept of storing packet timing when MPEG2-TS isrecorded on recording means 23. 511 a is an input TS from TSmultiplexing means 22 to recording means 23. 511 b is TS to be actuallyrecorded on a recording medium. 511 c is TS played and output from arecording medium. 300-302 are Null packets. 303-305 indicate timingwithout TS packets.

When an input TS of 511 a includes Null packets as shown in 300-302, theNull packets should be removed from a record TS of 511 b for recording,as the Null packets do not include information necessary for recordingand playing a picture and a sound. However, as arrival timing has timinginformation in PCR for MPEG2-TS, a correct STC is not played if it isplayed as in 511 b. Therefore, parts of Null packets need to be blankwith no packets as shown in 511 c or Null packets need to be insertedagain and input. In order to achieve this, 511 b adds ATS, whichindicates input timing for the packet, to the top of each packet to berecorded and record it.

FIG. 12 illustrates an example of a TS packet added with ATS. Four bytesof ATS are added before 188 bytes of TS packets for MPEG2-TS. Thisaddition of ATS is achieved by inputting a count from a counter of 32bits operating at 27 MHz for free run as ATS value generation means 90,for example, into ATS multiplexing means 91, and by adding a count wheneach TS packet is input into ATM recording means 23 before the TSpacket.

An example of a circuit for reproducing playing timing for a TS packetadded with ATS is shown in FIG. 13. Reference numeral 70 denotes asmoothing buffer, 71 denotes an ATS recorded on a smoothing buffer, 72denotes a comparator, 73 denotes a read-out controlling part, and 74denotes a counter.

A record TS shown in 511 b is input into smoothing buffer 70. Among TSpackets recorded in smoothing buffer 70, ATS 71, which is added to a TSpacket recorded first, is read out and input into comparator 72. On theother hand, counter 74 operates at 27 MHz clock, the same frequency asused for generating an ATS at recording, and outputs the count tocomparator 72.

When a record TS shown in 511 b is played, an ATS added to a TS packetto be played first is loaded to counter 74, while the first TS packet isoutput. Then, comparator 72 compares a value for ATS 71 added to afollowing TS packet with a value from counter 74. When the values match,comparator 72 notifies read-out controlling part 73 of matchinginformation. When the matching information is input from comparator 72,read-out controlling part 73 performs controlling so that the TS packetsadded with ATS 71 is read out from smoothing buffer 70 at that time.With these operations, the TS packets read out from smoothing buffer 70are output at the same timing as an input TS (511 a) as shown in 511 c.As the part, 303-305 shown in 511 c, has no TS packet added withcorresponding ATS, it will be blank without any data.

Effects of an ATS at usual recording and playing, i.e., when imagingframe-rate value X and playing frame-rate value Y are the same, havebeen described. For example, when a recording medium is memory, anaddress has to be specified to access there. For this reason, timingbetween packets cannot be stored for recording and playing if an inputtransport stream is used as it is. Therefore, timing between packets atinput needs to be reproduced in a system such as an ATS.

Now, configuration and operations of the present invention when imagingframe-rate value X and playing frame-rate value Y are set to differentvalues will be described. Here, it is assumed that playing frame-rateY=(24/1.001) fps. It is also assumed that reference data-rate ofMPEG2-TS at playing Rts=24 Mbps.

FIG. 14 is a timing chart illustrating operations of respective partswhen imaging frame-rate value X=(8/1.001) fps. 514 a is a video PES on arecording side output from PES-making means 50. PTS/DTS is added to avideo PES on a recording side by each piece of data compressing oneframe of picture signals at imaging frame-rate of eight fps (validframe-rate) Therefore, an interval for adding PTS/DTS reflects theimaging frame-rate.

514 b is MPEG2-TS added with an ATS output from ATS multiplexing means91. Data on MPEG2-TS added with an ATS output from ATS multiplexingmeans 91 to recording media controlling part 92 is considered to betransmitted on an eight bit parallel signal line at 27 MHz clock. Avideo PES shown in 514 e is divided into pieces, and each piece isstored in payload of a TS packet (TSP).

A data-rate of MPEG2-TS shown in 514 b is what a reference data-rate atplaying Rts is multiplied by a ratio between imaging frame-rate X andplaying frame-rate Y, i.e., Rts·(X/Y)=24 Mbps·(8 fps/24 fps)=8 Mbps.

Therefore, as an amount of data of MPEG2-TS is little in comparison witha transmission capacity of eight bit parallel signal line at 27 MHzclock, each interval between TS packets is blanked with no data. Aninterval between TS packets at recording is 188·27 MHz/(8 Mbps/8)=5076,where Rts·(X/Y)=8 Mbps.

That is to say, when the frame-rate of input picture signals is Xframes/second, the preset frame-rate of a system stream at playing is Yframes/second, compressed picture data is subjected to compressionencoding so that a reference data-rate at playing of the compressedpicture data can be decoded at R bits/second, and a data-rate at playingof a system stream is Rts bits/second (where Rts is a real number ofRts>R), TS multiplexing part 22 performs multiplexing so that an outputdata-rate of a system stream is at Rts×(X/Y) bits/second as it recordsthe system stream in recording part 23. Even if the frame-rate of inputpicture signals is different from the frame-rate at playing like this,TS multiplexing part 22 performs multiplexing on compressed picture dataso that the compressed picture data virtually matches with a referencedata-rate at playing of a system stream, corresponding to a frame-rateat playing.

An ATS is added before each TS packet. As an ATS uses a value for thearrival time of a TS packet at recording counted at 27 MHz in the usualrecording and playing described above, a differential value betweensuccessive ATSs is 5076, the same as a TS packet interval at recording.However, when imaging frame-rate value X and playing frame-rate value Yare set to different values, a value added as an ATS needs to be setaccording to timing at playing. This will be detailed later.

514 c is MPEG2-TS, which is played from a recording medium andreproduces timing of an original TS packet. However, the timing is notexactly the same as that at recording. The data-rate is different by aratio between an imaging frame-rate and a playing frame-rate. An ATS isprimarily removed at this stage, but the embodiment is described using adrawing with an ATS remained for simplicity.

514 d is a video PES separated from MPEG2-TS in 514 c. Also in thisdrawing, a data-rate is different by a ratio between an imagingframe-rate and a playing frame-rate for a video PES on recording side(514 a).

In FIG. 14, a time reference multiplexed in a stream, i.e., values forATS, PCR, PTS, and DTS are all provided according to a playing side,i.e., 514 c and 514 d. A differentiate value between adjacent ATSs,determined by a TS packet interval at playing as mentioned above, is188·27 MHz/(24 Mbps/8)=1692 when Rts=2 Mbps.

That is to say, when the frame-rate of input picture signals is Xframes/second and the preset frame-rate of a system stream at playing isY frames/second, ATS value generation means 90 calculates an ATS so thatthe ATS is virtually X/Y times a difference between successive referenceATS values where an ATS value generated as X and Y are the same is areference ATS value.

This ATS value generation means 90 can be configured to calculatedifferential value 1692 from imaging frame-rate value X, playingframe-rate value Y, and a value of reference data-rate at playing Rts byusing software on a microcomputer, store the value into a register, andsequentially give a value added to each input TS packet. This will bedetailed with reference to FIG. 17 later.

ATS value generation means 90 can be configured in another way to inputa counter value of a counter operating at ATS reference clock as in theusual recording (27 MHz at usual recording) in ATS referencemultiplexing means 90. This can be achieved with a configuration where afrequency of ATS reference clock can vary by 27 MHz·(X/Y) For example,when Y=(24/1.001) fps and X=(8/1.001) fps, a frequency of ATS referenceclock is 27 MHz·(8/24)=9 MHz. As a configuration for varying an ATSreference clock, a configuration shown in FIG. 7 illustrated in theembodiment 1 according to the present invention can be used. An ATSreference clock of the embodiment 2 is an example of a clock accordingto the present invention.

That is to say, when the frame-rate of input picture signals is Xframes/second and the preset frame-rate of a system stream at playing isY frames/second, ATS value generation means 90 changes a frequency ofATS reference clock to X/Y times a reference value as X and Y are thesame and uses the changed ATS reference clock to generate an ATS value.When a frame-rate of input picture signals and a preset frame-rate of asystem stream at playing are different, ATS value generation means 90generates an ATS value by changing a frequency of ATS reference clockfrom a reference value.

A differential value between adjacent PCRs, which is determined by a PCRpacket at playing, is 1692·L when a PCR packet is inserted for every Lpackets, for example. A differential value between adjacent PTS and DTS,which is determined by a frame interval at playing, is ( 1/24 fps)·90kHz=3750 when Y=24 fps. An offset value between a PCR value and aPTS/DTS value is set so as to satisfy buffering complying with the MPEG2standard at timing of 514 c and 514 d.

As described with reference to FIG. 14, when an imaging frame-rate valueX=8 fps, a playing frame-rate value Y=24 fps, and a reference data-rateat playing Rts=24 Mbps, intervals between TS packets at recording andplaying are integers if counted by 27 MHz clock. Therefore, a value foran ATS can be given without problems. However, some imaging frame-rate Xmake an interval between TS packets at recording other than integer whenit counted by 27 MHz clock. An example of this kind will be describedwith reference to FIG. 15.

FIG. 15 is a timing chart illustrating operations of respective partswhen imaging frame-rate value X=14.985 fps. 515 e is MPEG2-TS afterplayed from a recording media and reproduces timing of an original TSpacket, being exactly the same as 514 c. When timing on a recording sideis calculated from 515 e based on a ratio between an imaging frame-rateand a playing frame-rate, MPEG2-TS added with an ATS output from ATSmultiplexing means 91 is like 515 d, and a video PES on a recording sideoutput from PES-making means 50 is like 515 c. An interval between TSpackets is calculated at 515 d as: 188·27 MHz/(24 Mbps·(14.985fps/24/1.001) fps)/8)=2707.2, generating a fraction.

Although an interval between TS packets actually have to be integer, ifit is rounded up to 2708 clock, for example, multiple intervals of PCRsbroadens for 515 d like 515 b. On the other hand, a video PES on arecording side output from PES-making means 50 is output at exactly thesame timing as timing determined by an imaging frame-rate like 515 a,i.e., the same as 515 c. As a result, synchronized relationship betweenPCR and PTS/DTS become irregular, which cannot be correctly played asusual 24 fps picture signals at playing.

FIG. 16 is a diagram illustrating operations of respective parts whenimaging frame-rate value X=14. 985 fps and a fraction of an intervalbetween TS packets is addressed. 516 a is a video PES on a recordingside output from PES-making means 50, being exactly the same as 515 a.516 b is MPEG2-TS that multiplexed a video PES of 516 a and added withan ATS. An interval between TS packets is considered as 2707 clock here.When an interval between TS packets at playing is calculated by a ratiobetween imaging frame-rate and a playing frame-rate, it is 1691.875clock as shown in 516 d. However, a value with such a fraction cannot beused as a differential value for an ATS.

Thus, as shown in 516 c, when an ATS is given assuming a differentialvalue for successive ATSs being 1692, differences from 1691.875 areaccumulated to eight packets, which is exactly for one clock. A fractionof an interval between TS packets can be addressed by making adifferential value 1691 for once for eight packets when an ATS is given.

FIG. 17 shows a configuration of ATS value generation means 90 forgiving such an ATS. Reference numeral 80 denotes a differential valueregister, 81 denotes an adder, 82 denotes a selector, 83 denotes an ATSvalue counter, 84 denotes an ATS value register, 85 denotes an adder, 86denotes a comparator, and 87 denotes a J counter that counts up to Jthen returns to 0. ATS value generation means 90 is configured so thateach of an ATS differential value set in differential value register 80,a value J set in J counter 87, and a value K to compared with an outputfrom J counter 87 at comparator 86 can be set from a microcomputer (notshown). When an operation corresponding to FIG. 16 is performed, 1691 isset to ATS differential value and 7 is set to each of values J and K.

ATS value counter 83 and J counter 87 are driven with a pulsesynchronizing to an input of a TS packet input to ATS multiplexing means91. J counter 87 sequentially counts up from zero each time a TS packetis input, and when it counts 7, it returns to zero when the next TSpacket is input. Comparator 86 compares an output from J counter withK=7. When the output is less than K, i.e., when the output is betweenzero and six, selector 82 selects an output from adder 81 (1691+1=1692).When the output from J counter 87 is 7, the same as K, selector 82selects an output from differential value register 80 (1691). Therefore,ATS counter 83 adds 1692 as a differential value for seven TS packetsinput among eight TS packet input and adds 1691 for one TS packet to avalue stored in ATS value register 84. By using an output value from ATScounter 83 as an ATS value to be added to a TS packet, an ATS can begiven at the time and a value shown in 516 c.

Although an ATS differential value is selected from values of 1691 and1692 in the embodiment, the value is not limited to them and can beselected from three or more values.

Here, it is assumed that the frame-rate at recording is X and theframe-rate at playing is Y. In order to calculate an ATS differentialvalue so that a difference of ATS values to be added to a correspondingTS packet for recording with a frame-rate as Y is X/Y times a differenceof ATS values to be added to successive TS packets at recording, only anATS needs to be calculated so that an average of selected valuesselected from a plurality of values as an ATS value to be added to acorresponding packet is X/Y times a difference of ATS values added to acorresponding TS packet as it is recorded at frame-rate Y. In thismanner, a fraction of an interval between TS packets can be addressed.

When the frame-rate of an input picture signals and the presetframe-rate of a recorded system stream are different like this, ATSvalue generation means 90 multiplexes an ATS corresponding to a presetframe-rate of a system stream at playing in a TS packet.

Thus, the embodiment 2 has provided a video recording apparatus that canplay at the same frame-rate (24 frames/second) and the same TS data-rateRts as the device recorded in usual manner without requiring any specialdevice. This is because the device enables a variable frame-raterecording even in a recording and playing system for playing TS packettiming with an ATS (Arrival Time Stamp) and performs compressionencoding and multiplexing to comply with a preset playing frame-rate (24frames/second).

The embodiments 1 and 2 have been described by using the case that a TSdata-rate at playing Rts and a standard rate for compressed picture data(playing bit_rate) R are constant regardless of imaging frame-rate value(valid frame-rate) X. However, values for Rts and R need not be constantand can be changed depending on a value for X.

That is to say, when the frame-rate of input picture signals isdifferent from the frame-rate at playing, picture encoding part 21changes a data-rate of compressed picture data at playing, whichcorresponds to a frame-rate at playing, based on a frame-rate of inputpicture signals.

For example, in order to provide 2.5 times slow motion for 60 fpsimaging as shown in FIG. 8, where a standard rate at a usual recordingis 9 Mbps, the device needs to perform encoding at 22.5 Mbps, which is2.5 times the 9 Mbps. MPEG2 MP@ML encoder can be used for a usualrecording, though, encoding speed at recording 60 fps goes beyond therange of MPEG2 MP@ML, which requires a costly sophisticated encoder.

When imaging frame-rate X=60 fps and playing frame-rate Y=24 fps,encoding at recording is performed at 15 Mbps by changing a standardrate at usual recording to 6 Mbps. Thus, encoding is available withinthe range of MPEG2 MP@ML.

With the above operations, a variable frame-rate recording can beprovided with a substantially low cost encoder in compensation forpicture quality degraded by lowering standard rate R. In this case,decoding is available as typical MPEG2-TS at standard rate R=6 Mbps.

The same processing is also available in the case of recording andplaying with an ATS shown in FIG. 16. That is to say, when theframe-rate of input picture signals is different from the frame-rate atplaying, TS multiplexing part 22 changes a data-rate of a system streamat playing, which corresponds to a frame-rate at playing, to aframe-rate of input picture signals.

For example, when imaging frame-rate X=60 fps and playing frame-rateY=24 fps where MPEG2-TS reference data-rate Rts for usual recording is24 Mbps, a data-rate for MPEG2-TS at playing is 24 Mbps·(60 fps/24fps)=60 Mbps. Even if processing at this high rate is difficult at acompression encoding part and a TS multiplexing part, this processingcan be easily performed by changing to Rts=16 Mbps when X=60 fps, whichmakes a data-rate for MPEG2-TS at recording 16 Mbp·(60 fps/24 fps)=40Mbps.

As mentioned above, by changing TS data-rate at playing Rts and astandard rate of compressed picture data R based on imaging frame-ratevalue X, a variable frame-rate recording is available with lower costconfiguration.

Although the embodiments of the present invention have been described byassuming that MPEG2-TS is used for multiplexing, multiplexing methodsare not limited to this and MPEG2-PS (Program Stream), for example, canbe used. Although the embodiments also have been described by assumingthat MPEG2 picture encoding is used for compression encoding of apicture, compression encoding methods are not limited to this and MPEG4video encoding, H.264 video encoding, compression encoding in DVC(Digital Video Cassette), for example, can be used.

The embodiments have been described by assuming that video recordingapparatus 2 is provided with valid frame retrieving part 20 and pictureencoding part 21, video recording apparatus 2 is not limited to this andvideo recording apparatus 2 may not include valid frame retrieving part20, picture encoding part 21, and sound encoding part 24. In this case,a device outside to video recording apparatus 2 is provided with validframe retrieving part 20, picture encoding part 21, and sound encodingpart 24. Xp picture signals 6 and valid frame flag 7 output from imagingdevice 1 are processed in this outside device and the device outputscompressed picture data. Then, video recording apparatus 2 inputs thecompressed picture data output from the outside device into TSmultiplexing part 22, which generates a system stream. Video recordingapparatus 2 may not perform compression encoding on input picturesignals by itself and may input compressed picture data subjected tocompression encoding and only perform multiplexing.

The embodiments have been described by assuming that it multiplexes timemanagement information defining times for decoding of PCR and the likewhen it records into a recording medium, though the embodiments are notlimited this. When the embodiments records on a recording medium, it mayrecord in the form of a frame-rate at recording X without multiplexingsuch time management information and change the picture signals recordedon the recording medium into picture signals at Y frames/second by usinga frame converter or the like at playing.

That is to say, operations in this case are performed below. First,imaging device 1 that can vary a frame-rate at recording images picturesignals. The picture signals imaged at picture encoding part 21 issubjected to compression encoding and the compressed picture data isoutput. The output compressed picture data is not processed at TSmultiplexing part 22. Instead, the recording part 23 directly recordsthe compressed picture data output from picture encoding part 21 on arecording medium. When recording part 23 records the compressed picturedata on a recording medium, it records the compressed picture data at aframe-rate at imaging. When the compressed picture data recorded on therecording medium is played, the compressed picture data is converted toa predetermined frame-rate, which is previously determined by frameconverter or the like.

When X is 12 and Y is 24, for example, operations continue like below.Picture signals are taken with imaging device 1 at 12 frames/second. Thetaken picture signals are subjected to compression encoding andcompressed picture data is generated. The compressed picture data isrecorded on a recording medium at 12 frames/second. Unlike theembodiments, the compressed picture data needs not to be multiplexedwith time management information defining times about decoding of PCR orthe like. When the compressed picture data recorded on the recordingmedium is played, the compressed data is converted to picture signals at24 frames/second with a frame converter and output.

Therefore, as compressed picture data is recorded on a recording mediumat a frame-rate of taken picture signals in this case, it is effectivein that more recording medium can be saved compared with conventionalvideo recording apparatus. This is because, conventional video recordingapparatus records redundant compressed picture data on a recordingmedium, as it first converts picture signals at imaging into aframe-rate of 60 frames/second, which can be displayed on EVF, thencompresses the signals within a frame and generates compressed picturedata, and records the data as a picture in a frame-rate of 60frames/second on a recording medium.

A program according to the present invention is a program cooperatingwith a computer for causing the computer to perform functions of all orsome means of the above mentioned video recording apparatus of presentinvention.

A recording medium according to the present invention is a recordingmedium carrying a program for causing a computer to perform functions ofall or some means (or device, element, etc.) of the above mentionedvideo recording apparatus of present invention, wherein the programreadable for a computer and readout from a computer performs thefunctions with the computer.

The term “some means” of the present invention described above refers toone or more means among the plurality of means.

The term “functions of means” of the present invention described aboverefers to all or some functions among the means.

A usage of a program of the present invention can be an aspect where theprogram is recorded on a recording medium readable for a computer andcooperates with the computer.

A usage of a program of the present invention can be an aspect where theprogram is transmitted on a transmission medium, read by a computer, andcooperates with the computer.

A data structure of the present invention includes a database, a dataformat, a data table, a data list, a type of data.

Recording media includes ROM and transmission media includes atransmission media such as Internet, light wave, radio wave, and soundwave.

A computer of the present invention mentioned above is not limited tosheer hardware such as CPU, and can include firmware, an OS, or even aperipheral device.

As mentioned above, a configuration of the present invention can beimplemented by software or hardware.

An picture recording medium of the present invention is useful as avideo camera, and particularly suitable for a video camera that providesa slow motion effect and a quick motion effect in a movie not bychanging the playback speed of the film but by a digital frame-rateconversion.

1. A video recording apparatus comprising: a multiplexing deviceoperable to multiplex time management information defining times fordecoding and compressed picture data generated by performing compressionencoding of input picture signals and operable to output a systemstream; and a recording device operable to record the system stream on arecording medium; wherein, when a recording frame-rate of the inputpicture signals is different from a preset playing frame-rate forplaying picture signals obtained after recording, playing, and decodingthe system stream, said multiplexing device multiplexes the timemanagement information corresponding to the preset playing frame-rateand the compressed picture data.
 2. The video recording apparatusaccording to claim 1, comprising a picture encoding device operable toperform compression encoding of the input picture signals and output thecompressed picture data.
 3. The video recording apparatus according toclaim 1, wherein the compressed picture data is generated by performinginterframe compression on some frames of the input picture signals. 4.The video recording apparatus according to claim 1, wherein thecompressed picture data is generated from the input picture signalsobtained from an imaging device that can vary a frame-rate duringimaging.
 5. The video recording apparatus according to claim 1, whereinsaid multiplexing device is operable to perform multiplexing complyingwith an MPEG standard, and wherein the time management informationincludes PCR or SCR used as a timing reference for decoding andoutputting, wherein PTS indicates a timing for outputting each decodedframe, and DTS indicates a timing for decoding.
 6. The video recordingapparatus according to claim 1, wherein said multiplexing device has aclock operable to generate the time management information, said clockhaving a frequency which can be varied from a reference value used whenthe recording frame-rate and the playing frame-rate are the same; andwherein when the recording frame-rate is different from the playingframe-rate, said multiplexing device changes a frequency of said clockfrom the reference value and uses the changed frequency to generate thetime management information.
 7. The video recording apparatus accordingto claim 6, wherein, when the recording frame-rate is X frames/second (Xis a real number) and the playing frame-rate is Y frames/second, saidmultiplexing device changes a frequency of said clock to X/Y times thereference value when X and Y are the same and uses the changed frequencyto generate the time management information.
 8. The video recordingapparatus according to claim 1, wherein, when the recording frame-rateis X frames/second (X is an real number) and the playing frame-rate is Yframes/second, said multiplexing device calculates the time managementinformation so that a difference of values between pieces of the timemanagement information is X/Y times a difference of values betweencorresponding pieces of the reference time management information,wherein time management information generated when X and Y are the sameis the reference time management information.
 9. The video recordingapparatus according to claim 1, wherein, when the recording frame-rateis different from the playing frame-rate, said multiplexing devicegenerates the timing reference information with an interval formultiplexing different from a reference interval for multiplexing,wherein the reference interval for multiplexing is an interval formultiplexing in case the recording frame-rate and the playing frame-rateare the same, for timing reference information (PCR) among the timemanagement information.
 10. The video recording apparatus according toclaim 9, wherein, when the recording frame-rate is X frames/second (X isan real number) and the playing frame-rate is Y frames/second, saidmultiplexing device generates timing reference information so that thetiming reference information (PCR) among the time management informationis X/Y times a reference interval for multiplexing, wherein thereference interval for multiplexing is an interval for multiplexing incase X and Y are the same.
 11. A video recording apparatus comprising: apicture encoding device operable to perform compression encoding ofinput picture signals and output the compressed picture data; amultiplexing device operable to multiplex other information and thecompressed picture data and output a system stream; and a recordingdevice operable to record the system stream on a recording medium;wherein said picture encoding device is operable to change a recordingdata-rate of the compressed picture data from a reference data-rate andperform compression encoding of the input picture signals when arecording frame-rate is different from a playing frame-rate, wherein thereference data-rate is a playing data-rate of the compressed image datawhen a recording frame-rate of the input picture signals and a playingframe-rate preset in playing picture signals obtained after recording,playing, and decoding the system stream are the same.
 12. The videorecording apparatus according to claim 11, wherein, when the recordingframe-rate is different from the playing frame-rate, said pictureencoding device performs compression encoding of the input picturesignals so that data-rate during playing of the compressed picture dataapproximately matches the reference data-rate.
 13. The video recordingapparatus according to claim 12, wherein, when the recording frame-rateis X frames/second (X is a real number), the playing frame-rate is Yframes/second, and the reference data-rate is R bits/second (R is a realnumber), said picture encoding device performs rate-controlling so thatthe input picture signals are subjected to compression encoding at adata-rate of R×(X/Y) bits/second.
 14. The video recording apparatusaccording to claim 13, wherein the system stream is a stream complyingwith an MPEG2 standard; and wherein said picture encoding device isoperable to set a picture_rate indicating an input picture frame-ratefor rate-controlling to X and a bit_rate indicating a target rate ofcompressed picture data to R×(X/Y), respectively.
 15. The videorecording apparatus according to claim 12, wherein, when the recordingframe-rate is X frames/second (X is a real number), the playingframe-rate is Y frames/second, and the reference data-rate is Rbits/second (R is a real number), said picture encoding device performsrate-controlling so as to perform compression encoding of picturesignals whose frame-rate is Y at a data-rate of R bits/second regardlessof a frame-rate of the input picture signals being X.
 16. The videorecording apparatus according to claim 15, wherein the system stream isa stream complying with an MPEG2 standard; and wherein said pictureencoding device is operable to set a picture_rate indicating an inputpicture frame-rate for rate-controlling to Y and a bit_rate indicating atarget rate of compressed picture data to R, respectively.
 17. The videorecording apparatus according to claim 11, wherein, when the recordingframe-rate is different from the playing frame-rate, said pictureencoding device changes a data-rate of the compressed picture dataduring playing, which corresponds to the playing frame-rate, based onthe recording frame-rate.
 18. A video recording apparatus comprising: amultiplexing device operable to multiplex other information andcompressed picture data generated by performing compression encoding onpicture signals and output a system stream; and a recording deviceoperable to record the system stream on a recording medium; wherein saidmultiplexing device is operable to change a recording system data-rateof the system stream from a reference data-rate and perform multiplexingwhen a recording frame-rate is different from a playing frame-rate,wherein the reference system data-rate is a playing data-rate of thesystem stream when a recording frame-rate of the input picture data anda preset playing frame-rate of playing picture signals obtained afterrecording, playing, and decoding of the system stream are the same. 19.The video recording apparatus according to claim 18, wherein, when therecording frame-rate is different from the playing frame-rate, saidmultiplexing device performs multiplexing of the compressed encodingpicture signals so that the data-rate of the system stream duringplaying, which corresponds to the playing frame-rate, approximatelymatches the reference system data-rate.
 20. The video recordingapparatus according to claim 19, wherein said multiplexing device isoperable to perform the multiplexing so that an output data-rate of thesystem stream is at Rts×(X/Y) bits/second as said multiplexing deviceoutputs the system stream to said recording device when the compressedpicture data has been subjected to compression encoding so that thecompressed picture data can be decoded at R bits/second even if X isdifferent from Y and a data-rate of the system stream is Rts bits/second(Rts is a real number of Rts>R), wherein the recording frame-rate is Xframes/second (X is a real number), the playing frame-rate is Yframes/second, and a playing data-rate of the compressed picture datawhen X and Y are the same is reference data-rate R bits/second (R is areal number).
 21. The video recording apparatus according to claim 18,wherein, when the recording frame-rate is different from the playingframe-rate, said multiplexing device changes a data-rate of the systemstream during playing based on a frame-rate of the input picturesignals.
 22. The video recording apparatus according to claim 2,comprising a valid frame detection device operable to retrieve a validframe flag from the input picture signals by using an input valid frameflag and output the retrieved valid frame to said picture encodingdevice when the input picture signals are picture signals generated byrepeating each frame one or more times and when a valid frame flagindicating a valid frame, which is an original frame for generating saidinput picture signals, is input.
 23. The video recording apparatusaccording to claim 1, wherein said multiplexing device is operable toinput sound data in addition to the compressed picture data, and (1)when the recording frame-rate and the playing frame-rate are the same,said multiplexing device multiplexes the input sound data and the systemstream, and (2) when the recording frame-rate is different from theplaying frame-rate, said multiplexing device does not multiplex theinput sound data and the system stream.
 24. The video recordingapparatus according to claim 1, comprising a sound conversion device,wherein, when sound data is also input in addition to the compressedpicture data, and (1) when the recording frame-rate and the playingframe-rate are the same, said sound conversion device outputs the sounddata as recorded sound data without changing a sampling frequency of theinput sound data, and (2) when the recording frame-rate is differentfrom the playing frame-rate, said sound conversion device converts thesampling frequency of the input sound data to a different frequency andoutputs the sound data as recorded sound data; and wherein saidmultiplexing device is operable to multiplex the output recorded sounddata in the system stream.
 25. The video recording apparatus accordingto claim 24, wherein said sound conversion device is operable to converta sampling frequency of the input sound data to X/Y times the frequencywhen X and Y are different, wherein the recording frame-rate is Xframes/second (X is a real number) and the playing frame-rate is Yframes/second (Y is a real number).
 26. The video recording apparatusaccording to claim 1, wherein the system stream comprises a plurality ofpackets; and wherein said recording device is operable to multiplexpacket playing time management information defining times for playing inthe packets included in the system stream, and record it on therecording medium; wherein said recording device is operable to changepacket playing time information from reference packet playing timeinformation and multiplex the information when the recording frame-rateis different from the playing frame-rate; wherein the reference packetplaying time information is packet playing time information when therecording frame-rate and the playing frame-rate are the same.
 27. Thevideo recording apparatus according to claim 26, wherein said recordingdevice has a clock operable to generate the packet playing timeinformation whose frequency can be changed from a reference value usedwhen the recording frame-rate and the playing frame-rate are the same;and wherein, when the recording frame-rate is different from the playingframe-rate, said recording device changes a frequency of said clock fromthe reference value and uses the changed frequency to generate thepacket playing time management information.
 28. The video recordingapparatus according to claim 27, wherein, when the recording frame-rateis X frames/second (X is a real number) and the playing frame-rate is Yframes/second, said recording device changes a frequency of said clockto X/Y times the reference value when X and Y are the same and uses saidclock to generate the packet playing time management information. 29.The video recording apparatus according to claim 26, wherein, when therecording frame-rate is X frames/second (X is an real number) and theplaying frame-rate is Y frames/second, said recording device calculatesthe packet playing time information so that a difference between valuesfor successive packet playing time information is approximately X/Ytimes a difference between values for the corresponding reference packetplaying time information.
 30. The video recording apparatus according toclaim 29, wherein said recording device is operable to determine byselecting a difference of corresponding packet playing time informationfrom a plurality of values and calculate the packet playing timeinformation so that an average of the selected results is X/Y times adifference of the reference packet playing time information when saidrecording device calculates the difference between values ofcorresponding packet playing time information so that the packet playingtime information is approximately X/Y times a difference of successivepieces of the reference packet playing time information.
 31. A videorecording apparatus comprising: a picture encoding device operable toperform compression encoding of input picture signals obtained from animaging device that can vary a frame-rate during imaging and outputcompressed picture data; and a recording device operable to record theoutputted compressed picture data on a recording medium; wherein thecompressed picture data is recorded on the recording medium at theframe-rate during imaging; and wherein the compressed picture datarecorded on the recording medium is converted into a previously decidedpredetermined frame-rate for playing.
 32. A multiplexing methodcomprising: multiplexing time management information defining times fordecoding and compressed picture data generated by performing compressionencoding on picture signals and outputting a system stream; andrecording the system stream on a recording medium, wherein, when arecording frame-rate of the input picture signals is different from apreset playing frame-rate for playing picture signals obtained afterrecording, playing, and decoding the system stream, said multiplexingmultiplexes the time management information corresponding to the presetplaying frame-rate and the compressed picture data.
 33. A multiplexingmethod for multiplexing time management information defining times fordecoding and compressed picture data that has been subjected tocompression encoding on X picture signals whose frame-rate is Xframes/second (X is a real number) so that the X picture signals can bedecoded as Y picture signals whose frame-rate is Y frames/second (Y is areal number) at a playing bit rate of R bits/second (R is a realnumber)□ wherein said method comprises multiplexing standard timemanagement information so that a difference between values of timemanagement information is X/Y-fold; and wherein when X=Y, a referencetime management information is time management information to bemultiplexed on compressed picture data that has been subjected tocompression encoding so that the X picture signals can be decoded atplaying bit rate R bits/second.
 34. A multiplexing method formultiplexing time management information defining times for decoding andcompressed picture data that has been subjected to compression encodingso that X picture signals whose frame-rate is X frames/second (X is areal number) can be decoded as Y picture signals whose frame-rate is Yframes/second (Y is a real number); wherein said method comprisescontrolling timing reference information (PCR) among time managementinformation so that an interval for multiplying is Y/X-fold.
 35. Apicture encoding method comprising: performing compression encoding oninput picture signals and outputting the compressed picture data;multiplexing other information and the compressed picture data andoutputting a system stream; and recording the system stream on arecording medium; wherein, said performing compression encoding changesa recording data-rate of the compressed picture data from a referencedata-rate and performs compression encoding on the input picture signalswhen the recording frame-rate is different from the playing frame-rate,wherein a reference data-rate is a playing data-rate of the compressedimage data when a recording frame-rate of the picture signals and apreset playing frame-rate of playing picture signals obtained afterrecording, playing, and decoding the system stream are the same.
 36. Apicture encoding method for performing compression encoding X picturesignals whose frame-rate is X frames/second (X is a real number) so thatthe X picture signals can be decoded as Y picture signals whoseframe-rate is Y frames/second (Y is a real number) at a playing bit rateR bits/second (R is a real number), said method comprising: obtaining aplaying frame-rate Y and a playing bit-rate R; inputting a value forimaging a frame-rate X; setting an input picture frame-rate for arate-controlling picture_rate to X and a target rate of a compressedpicture data bit_rate to R·(X/Y); and performing rate-controlling ofcompression encoding by using the values for the picture_rate andbit_rate.
 37. A picture encoding method for performing compressionencoding on X picture signals whose frame-rate is X frames/second (X isa real number) so that the X picture signals can be decoded as Y picturesignals whose frame-rate is Y frames/second (Y is a real number) at aplaying bit rate R bits/second (R is a real number), said methodcomprising: obtaining a playing frame-rate Y and a playing bit rate Rfrom outside or a memory; inputting a value for imaging a frame-rate X;setting an input picture frame-rate for a rate-controlling picture_rateto Y and a target rate of a compressed picture data bit_rate to R; andperforming rate-controlling on compression encoding by using values forthe picture_rate and bit_rate.
 38. A program for use with a computer,said program comprising: an executable code operable to cause thecomputer to multiplex time management information defining times fordecoding and compressed picture data generated by performing compressionencoding of picture signals and cause the computer to output a systemstream; and an executable code operable to cause the computer to recordthe system stream on a recording medium; wherein, when a recordingframe-rate of the input picture signals is different from a presetplaying frame-rate for playing picture signals obtained after recording,playing, and decoding the system stream, said executable code operableto cause the computer to multiplex multiplexes the time managementinformation corresponding to the preset playing frame-rate and thecompressed picture data.
 39. A program for use with a computer, saidprogram comprising: an executable code operable to cause the computer toperforming compression encoding of input picture signals and output thecompressed picture data; an executable code operable to cause thecomputer to multiplex other information and the compressed picture dataand output a system stream; and an executable code operable to cause thecomputer to record the system stream on a recording medium; and whereinsaid executable code operable to cause the computer to performingcompression encoding is operable to change a recording data-rate of thecompressed picture data from a reference data-rate and performcompression encoding of the input picture signals when a recordingframe-rate is different from a playing frame-rate, wherein the referencedata-rate is a playing data-rate of the compressed image data when arecording frame-rate of the input picture signals and a playingframe-rate preset in playing picture signals obtained after recording,playing, and decoding the system stream are the same.
 40. A program foruse with a computer, said program comprising: an executable codeoperable to cause the computer to multiplex other information andcompressed picture data generated by performing compression encoding onpicture signals and cause the computer to output a system stream; and anexecutable code operable to cause the computer to record the systemstream on a recording medium; wherein said executable code operable tocause the computer to multiplex is operable to change a recording systemdata-rate of the system stream from a reference data-rate and performmultiplexing when a recording frame-rate is different from a playingframe-rate, wherein the reference system data-rate is a playingdata-rate of the system stream when a recording frame-rate of the inputpicture data and a preset playing frame-rate of playing picture signalsobtained after recording, playing, and decoding of the system stream arethe same.
 41. A computer readable medium containing a program accordingto claim
 38. 42. A computer readable medium containing a programaccording to claim
 39. 43. A computer readable medium containing aprogram according to claim
 40. 44. The video recording apparatusaccording to claim 11, wherein said multiplexing device is operable toinput sound data in addition to the compressed picture data, and (1)when the recording frame-rate and the playing frame-rate are the same,said multiplexing device multiplexes the input sound data and the systemstream, and (2) when the recording frame-rate is different from theplaying frame-rate, said multiplexing device does not multiplex theinput sound data and the system stream.
 45. The video recordingapparatus according to claim 18, wherein said multiplexing device isoperable to input sound data in addition to the compressed picture data,and (1) when the recording frame-rate and the playing frame-rate are thesame, said multiplexing device multiplexes the input sound data and thesystem stream, and (2) when the recording frame-rate is different fromthe playing frame-rate, said multiplexing device does not multiplex theinput sound data and the system stream.
 46. The video recordingapparatus according to claim 11, comprising a sound conversion device,wherein, when sound data is also input in addition to the compressedpicture data, and (1) when the recording frame-rate and the playingframe-rate are the same, said sound conversion device outputs the sounddata as recorded sound data without changing a sampling frequency of theinput sound data, and (2) when the recording frame-rate is differentfrom the playing frame-rate, said sound conversion device converts thesampling frequency of the input sound data to a different frequency andoutputs the sound data as recorded sound data; and wherein saidmultiplexing device is operable to multiplex the output recorded sounddata in the system stream.
 47. The video recording apparatus accordingto claim 18, comprising a sound conversion device, wherein, when sounddata is also input in addition to the compressed picture data, and (1)when the recording frame-rate and the playing frame-rate are the same,said sound conversion device outputs the sound data as recorded sounddata without changing a sampling frequency of the input sound data, and(2) when the recording frame-rate is different from the playingframe-rate, said sound conversion device converts the sampling frequencyof the input sound data to a different frequency and outputs the sounddata as recorded sound data; and wherein said multiplexing device isoperable to multiplex the output recorded sound data in the systemstream.